Changes needed to support ONIX #360

chrishalcrow · 2025-09-01T11:26:49Z

This PR shows what changes would be needed to implement ONIX support directly using read_openehys. This would avoid having to make a separate .json file and you get mux tables etc for free! It's a messy PR, just to see what the pain points are. I've written comments where I'm confused/unsure about something.

Test data: open-ephys-plugins/onix-source#144 (comment)

for more information, see https://pre-commit.ci

src/probeinterface/neuropixels_tools.py

chrishalcrow · 2025-09-01T11:27:51Z

src/probeinterface/neuropixels_tools.py

        custom_parameters = editor.find("CUSTOM_PARAMETERS")
-        np_probes = custom_parameters.findall("NP_PROBE")
+        if onix_processor is not None:
+            np_probes = custom_parameters.findall("NEUROPIXELSV1E")


Will NEUROPIXELSV1E always be the name of the probe?

I think it might be V1 or V2 depeneding on Neuropixels1.0/2.0. @bparks13 could you confirm?

Currently the available options are NEUROPIXELSV1E, NEUROPIXELSV1F, and NEUROPIXELSV2E. The V1E/V1F are both Neuropixels 1.0 probes, and can be used interchangeably at this stage, the E/F distinction is differentiating the other ONIX hardware/headstage.

codecov · 2025-09-01T11:28:21Z

Codecov Report

❌ Patch coverage is 83.80952% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.11%. Comparing base (d385af8) to head (92c8d05).
⚠️ Report is 26 commits behind head on main.

Files with missing lines	Patch %	Lines
src/probeinterface/neuropixels_tools.py	83.80%	17 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #360      +/-   ##
==========================================
+ Coverage   89.92%   90.11%   +0.18%     
==========================================
  Files          12       12              
  Lines        2044     2083      +39     
==========================================
+ Hits         1838     1877      +39     
  Misses        206      206

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

chrishalcrow · 2025-09-01T11:29:13Z

src/probeinterface/neuropixels_tools.py

+        if onix_processor is not None:
+            slot, port, dock = None, None, None
+            probe_part_number, probe_serial_number = np_probe.attrib["probePartNumber"], np_probe.attrib["probeSerialNumber"]
+            channels = np_probe.find("SELECTED_CHANNELS")


Here, the names of several settings are slightly different than the other cases. e.g. ONIX uses probePartNumber instead of probe_part_number.
Also, I don't think ONIX stores the slot, port and dock? Or maybe PortB in the probe_names_used says which port is used?

In terms of naming, some of these elements have been introduced in the draft PR here, which gives us some flexibility with changing the names. I could modify the name to be probe_part_number if that makes more sense to be consistent.

Since we are recording from Open Ephys hardware, and not IMEC hardware, we do not have a slot/port/dock to report. The Port in this instance is specifying which port on the Breakout Board the headstage is connected to.

chrishalcrow · 2025-09-01T11:29:40Z

src/probeinterface/neuropixels_tools.py

+            xpos = np.array([float(electrode_xpos.attrib[ch]) for ch in channel_names])
+            ypos = np.array([float(electrode_ypos.attrib[ch]) for ch in channel_names])
+            positions = np.array([xpos, ypos]).T
+        else:


This seems like the hardest thing: electrode positions are not saved with ONIX, but they are for the other formats.

yep, I think that the easiest would be to store the electrode x/y positions, or the selected electrods. In the latter case, we could instantiate a full NP probe from the probe part number and slice it accordingly.

We are currently storing the selected electrodes in the settings.xml file, under the SELECTED_CHANNELS node. The indices are the global electrode index. I would be happy to talk through how it is currently set up, and we can determine if that information is enough or I can add in more details as needed.

bparks13 · 2025-09-11T19:41:23Z

@chrishalcrow @alejoe91 I have added some additional zipped folders containing some more diverse electrode configurations in the plugin PR: NPX 1.0 configurations NPX 2.0 configurations

As per our discussion, you should not need the JSON files that are in the zipped folders, but the settings.xml files contain the selected electrode index and the probe part number, which should be sufficient. The JSON files will still be kept there for ease of use for the user, but are not needed in this PR.

Let me know if you need anything else from me!

chrishalcrow · 2025-09-12T14:01:17Z

Hey @bparks13 - thanks for all the new data. So when you have one probe the structure is e.g.

<NEUROPIXELSV1E>
    <PROBEVIEWER>

and the probe information is stored in the NEUROPIXELSV1E tag.

And when you have two or more it's e.g.

<NEUROPIXELSV2E>
    <PROBE0>
        <PROBEVIEWER>
        ...
    <PROBE1>
        <PROBEVIEWER>
        ...
    <PROBE2>
        <PROBEVIEWER>
        ...

and the probe information is stored in the PROBE0 etc. tags.

Is that right?

Are the names NEUROPIXELSV1E and NEUROPIXELSV2E related to the number of probes? Or just the NP version?

bparks13 · 2025-09-12T14:32:54Z

@chrishalcrow The names are related to the NP version, as you mentioned.

The difference in the structure is related to the physical hardware we use. Some headstages (NeuropixelsV1e/NeuropixelsV1f) have one probe per device, while NeuropixelsV2e headstages have one device with two probes:

NeuropixelsV1e Headstage (link)
- Can support a single Neuropixels 1.0 probe
- Probe information is stored in the NEUROPIXELSV1E tag, and SELECTED_CHANNELS is a sub-field
NeuropixelsV1f Headstage (link)
- Can support up to two Neuropixels 1.0 probes in either position (Probe0 / Probe1)
- Probe information is stored in the NEUROPIXELSV1F tag, and SELECTED_CHANNELS is a sub-field
  - There are two NEUROPIXELSV1F tags, differentiated by their name / probe serial number. The corresponding stream will have either Probe0 or Probe1 in the stream name.
  - If there is no serial number, then that probe is not connected.
- Here is a sample XML file, this does not have the probe serial number or the stream names, but hopefully this makes sense in how the structure is laid out
  - npx1f.xml
NeuropixelsV2e Headstage (link)
- Can support up to two Neuropixels 2.0 probes in either position (Probe0 / Probe1)
- Probe information is nested under the NEUROPIXELSV2E tag, where we now have PROBE0/PROBE1 tags with the probe information saved there, and within each of those tags is the SELECTED_CHANNELS sub-field.

I hope this helps! Let me know if anything doesn't make sense. I'm happy to meet and discuss it in more detail as well.

chrishalcrow · 2025-09-12T15:40:32Z

Amazing - very helpful!

bparks13 · 2025-10-08T19:37:15Z

@alejoe91 Here is a link where you can find the configurations with a single-shank probe, that correctly stays within its electrode boundaries!

open-ephys-plugins/onix-source#152 (comment)

Let me know if you need anything else, but it looks like once these are added to the tests it should be good to go.

src/probeinterface/neuropixels_tools.py

alejoe91 · 2025-10-22T08:34:10Z

@chrishalcrow @bparks13 can we merge this? LGTM now, but let's have a final look

Co-authored-by: Chris Halcrow <[email protected]>

bparks13 · 2025-10-23T20:21:44Z

@alejoe91 everything looks good to me, the only thing I want to confirm is if the inter_sample_shifts are correctly added to the probe/recording metadata with these changes? I've had a user ask me about that, and I just want to confirm. I assume it does, since these changes pull in the ProbeTable information, but wanted to check before we merge.

@chrishalcrow also pointed out a dtype issue above that should be addressed, and I believe once that is done we are good to go!

alejoe91 · 2025-10-24T07:51:48Z

@alejoe91 everything looks good to me, the only thing I want to confirm is if the inter_sample_shifts are correctly added to the probe/recording metadata with these changes? I've had a user ask me about that, and I just want to confirm. I assume it does, since these changes pull in the ProbeTable information, but wanted to check before we merge.

@chrishalcrow also pointed out a dtype issue above that should be addressed, and I believe once that is done we are good to go!

Yes! This is already the case. We recently refactored the mechanism because there was an issue when the user chose to save a subset of channels. But it will be fixed in the next release.

For reference, this is the function that gets sample shifts from the probe: https://github.com/SpikeInterface/spikeinterface/blob/main/src/spikeinterface/extractors/neuropixels_utils.py#L10

chrishalcrow and others added 2 commits September 1, 2025 12:25

changes needed for onix

30cf601

[pre-commit.ci] auto fixes from pre-commit.com hooks

8669835

for more information, see https://pre-commit.ci

chrishalcrow commented Sep 1, 2025

View reviewed changes

src/probeinterface/neuropixels_tools.py Show resolved Hide resolved

chrishalcrow commented Sep 1, 2025

View reviewed changes

chrishalcrow mentioned this pull request Sep 1, 2025

Proposal: Add ProbeInterface Support for ONIX Source Plugin #358

Open

chrishalcrow and others added 2 commits September 28, 2025 19:39

further onix fun

37dbf9c

Clean up ONIX reader for openeohys and add tests

ececc17

alejoe91 marked this pull request as ready for review October 1, 2025 14:08

alejoe91 added 2 commits October 7, 2025 16:26

Add NP2 onix data and support SELECTED_ELECTRODES

feaa386

Only support SELECTED_ELECTRODES, not SELECTED_CHANNELS

c52aee1

alejoe91 mentioned this pull request Oct 7, 2025

Automatically save Probe Interface files when recording starts open-ephys-plugins/onix-source#144

Open

Add more NP2 onix test files

35c5a2d

chrishalcrow commented Oct 14, 2025

View reviewed changes

src/probeinterface/neuropixels_tools.py Outdated Show resolved Hide resolved

chrishalcrow commented Oct 14, 2025

View reviewed changes

src/probeinterface/neuropixels_tools.py Outdated Show resolved Hide resolved

chrishalcrow commented Oct 14, 2025

View reviewed changes

src/probeinterface/neuropixels_tools.py Show resolved Hide resolved

Update src/probeinterface/neuropixels_tools.py

2ffd8cf

Co-authored-by: Chris Halcrow <[email protected]>

Make elec_ids and shank_ids int

92c8d05

alejoe91 merged commit c7901ed into SpikeInterface:main Oct 24, 2025
5 of 6 checks passed

Changes needed to support ONIX #360

Changes needed to support ONIX #360

Uh oh!

Conversation

chrishalcrow commented Sep 1, 2025 • edited by alejoe91 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bparks13 commented Sep 11, 2025

Uh oh!

chrishalcrow commented Sep 12, 2025

Uh oh!

bparks13 commented Sep 12, 2025

Uh oh!

chrishalcrow commented Sep 12, 2025

Uh oh!

bparks13 commented Oct 8, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alejoe91 commented Oct 22, 2025

Uh oh!

bparks13 commented Oct 23, 2025

Uh oh!

alejoe91 commented Oct 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

chrishalcrow commented Sep 1, 2025 •

edited by alejoe91

Loading

codecov bot commented Sep 1, 2025 •

edited

Loading